1
00:00:07,720 --> 00:00:05,170
earlier today Tom Marshburn spent a good

2
00:00:10,720 --> 00:00:07,730
bit of time assisting with a ground

3
00:00:14,049 --> 00:00:10,730
commanded activity saw a couple of small

4
00:00:16,590 --> 00:00:14,059
satellites flying inside the station to

5
00:00:20,410 --> 00:00:16,600
construct a 3d model of another object

6
00:00:22,120 --> 00:00:20,420
the spheres experiment is familiar to

7
00:00:23,860 --> 00:00:22,130
those who've been following the

8
00:00:25,840 --> 00:00:23,870
International Space Station program it

9
00:00:29,109 --> 00:00:25,850
got started on the space station during

10
00:00:32,470 --> 00:00:29,119
expedition 8 but today's activity was

11
00:00:35,290 --> 00:00:32,480
part of the spheres vertigo experiment

12
00:00:37,960 --> 00:00:35,300
in this case vertigo stands for visual

13
00:00:41,020 --> 00:00:37,970
estimation and relative tracking for

14
00:00:42,639 --> 00:00:41,030
inspection of generic objects and to

15
00:00:44,740 --> 00:00:42,649
learn more about this project I'm joined

16
00:00:47,170 --> 00:00:44,750
now by Brent twiddle he's a PhD

17
00:00:49,780 --> 00:00:47,180
candidate at the space systems

18
00:00:51,670 --> 00:00:49,790
laboratory at MIT in Cambridge

19
00:00:53,680 --> 00:00:51,680
Massachusetts he's a member of the

20
00:00:56,139 --> 00:00:53,690
spheres of vertigo team that ran this

21
00:00:57,850 --> 00:00:56,149
morning's test run and is in fact the

22
00:01:00,250 --> 00:00:57,860
man who was on the radio talking to Tom

23
00:01:01,930 --> 00:01:00,260
Marshburn during the activity Brent good

24
00:01:04,630 --> 00:01:01,940
morning was this your first time talking

25
00:01:06,609 --> 00:01:04,640
to astronauts in space good morning yeah

26
00:01:08,679 --> 00:01:06,619
actually this was it was an early

27
00:01:10,629 --> 00:01:08,689
morning for us and but it was really

28
00:01:12,249 --> 00:01:10,639
great to talk to Tom and see him work

29
00:01:14,980 --> 00:01:12,259
with our hardware for the first time on

30
00:01:17,529 --> 00:01:14,990
orbit tell me how you got involved in

31
00:01:19,749 --> 00:01:17,539
the vertigo project so I've actually

32
00:01:22,269 --> 00:01:19,759
been involved for a good number of years

33
00:01:24,279 --> 00:01:22,279
it started back in when I first came

34
00:01:27,340 --> 00:01:24,289
into grad school my advisor professor

35
00:01:28,899 --> 00:01:27,350
David Miller sort of said to me look

36
00:01:30,669 --> 00:01:28,909
we've got these spheres satellites that

37
00:01:33,669 --> 00:01:30,679
are on the International Space Station

38
00:01:36,999 --> 00:01:33,679
and we want to put put cameras on them

39
00:01:39,669 --> 00:01:37,009
and go so sort of a blank slate that I

40
00:01:41,289 --> 00:01:39,679
had to go through and we had a little

41
00:01:44,349 --> 00:01:41,299
bit of funding we were partnered with

42
00:01:46,839 --> 00:01:44,359
the naval research lab in Washington DC

43
00:01:48,849 --> 00:01:46,849
and we built up a good ground prototype

44
00:01:51,190 --> 00:01:48,859
of this and tested it out with a couple

45
00:01:52,989 --> 00:01:51,200
of visual navigation algorithms then

46
00:01:55,149 --> 00:01:52,999
after that program actually was quite

47
00:01:58,389 --> 00:01:55,159
successful we got some follow-on funding

48
00:02:00,399 --> 00:01:58,399
from the the folks at DARPA to take the

49
00:02:02,649 --> 00:02:00,409
hardware that we built and transition it

50
00:02:03,580 --> 00:02:02,659
to to an operational system that could

51
00:02:06,429 --> 00:02:03,590
actually be launched to the

52
00:02:08,440 --> 00:02:06,439
international space station so that was

53
00:02:10,300 --> 00:02:08,450
a big part of my PhD and we formed a

54
00:02:12,880 --> 00:02:10,310
team with a whole bunch of other sort of

55
00:02:14,370 --> 00:02:12,890
organizations Aurora flight sciences in

56
00:02:16,710 --> 00:02:14,380
Cambridge Massachusetts

57
00:02:19,460 --> 00:02:16,720
and then we were integrated by stp and

58
00:02:22,590 --> 00:02:19,470
NASA Ames into the into the NASA process

59
00:02:24,300 --> 00:02:22,600
and there are a lot of grad students and

60
00:02:26,190 --> 00:02:24,310
undergrads that join the team and we

61
00:02:28,490 --> 00:02:26,200
sort of put everything together shipped

62
00:02:31,710 --> 00:02:28,500
our hardware I think it was last August

63
00:02:33,690 --> 00:02:31,720
August of last year in 2012 and it

64
00:02:36,200 --> 00:02:33,700
launched in october and today was our

65
00:02:39,920 --> 00:02:36,210
first operations that went very well i

66
00:02:42,180 --> 00:02:39,930
want to get you to tell me about how so

67
00:02:43,770 --> 00:02:42,190
explain for those who was who are

68
00:02:46,530 --> 00:02:43,780
familiar with seeing these spheres

69
00:02:49,380 --> 00:02:46,540
satellites commanded to to fly in

70
00:02:51,420 --> 00:02:49,390
formation onboard the station how is the

71
00:02:54,240 --> 00:02:51,430
vertigo experiment differ from what

72
00:02:56,640 --> 00:02:54,250
we've seen before right so spheres is

73
00:03:00,150 --> 00:02:56,650
system as you said for formation flying

74
00:03:02,220 --> 00:03:00,160
but it relies solely on a set of

75
00:03:04,650 --> 00:03:02,230
ultrasonic beacons that are installed

76
00:03:07,770 --> 00:03:04,660
inside the International Space Station

77
00:03:09,750 --> 00:03:07,780
to figure out where it is within within

78
00:03:11,130 --> 00:03:09,760
the crew volume and one of the things

79
00:03:12,380 --> 00:03:11,140
that we're looking at doing and sort of

80
00:03:15,720 --> 00:03:12,390
opening a whole bunch of research

81
00:03:17,790 --> 00:03:15,730
possibilities is to add a pair of stereo

82
00:03:20,820 --> 00:03:17,800
cameras a much sort of like the human

83
00:03:23,160 --> 00:03:20,830
eye set of stereo or the stereo set of

84
00:03:25,950 --> 00:03:23,170
human eyes to actually be able to see

85
00:03:29,040 --> 00:03:25,960
perceive and understand its world

86
00:03:31,320 --> 00:03:29,050
visually and we use that information and

87
00:03:33,630 --> 00:03:31,330
communicate that information to the

88
00:03:35,910 --> 00:03:33,640
spheres satellites using using a package

89
00:03:38,340 --> 00:03:35,920
that's called the the vertigo goggles

90
00:03:41,970 --> 00:03:38,350
and these goggles are they're sort of a

91
00:03:43,949 --> 00:03:41,980
1.6 kilogram they're almost like a

92
00:03:46,860 --> 00:03:43,959
netbook computer in terms of they have

93
00:03:49,440 --> 00:03:46,870
processing power cameras Wi-Fi device a

94
00:03:52,080 --> 00:03:49,450
battery on board and their own their own

95
00:03:53,790 --> 00:03:52,090
sort of little intelligence block that

96
00:03:55,410 --> 00:03:53,800
sticks on the front end of the of the

97
00:03:57,090 --> 00:03:55,420
spheres satellites and allows it to see

98
00:03:59,699 --> 00:03:57,100
the rest of the world that it wants to

99
00:04:01,680 --> 00:03:59,709
navigate through you provided us some

100
00:04:04,710 --> 00:04:01,690
pictures of that equipment that was used

101
00:04:07,410 --> 00:04:04,720
today including the hardware that you've

102
00:04:09,540 --> 00:04:07,420
just described it it for all the world

103
00:04:12,180 --> 00:04:09,550
looks like you put eyes on the front of

104
00:04:13,980 --> 00:04:12,190
this spheres all right yeah and it's

105
00:04:15,300 --> 00:04:13,990
it's sort of funny the way that kind of

106
00:04:18,000 --> 00:04:15,310
fell out I mean we weren't trying to

107
00:04:19,560 --> 00:04:18,010
make it look like anything but a lot of

108
00:04:22,230 --> 00:04:19,570
people have commented it kind of looks

109
00:04:23,820 --> 00:04:22,240
like a like a wall-e figure but it

110
00:04:26,390 --> 00:04:23,830
really just fell out of the requirements

111
00:04:28,129 --> 00:04:26,400
of being able to see

112
00:04:31,280 --> 00:04:28,139
build a three-dimensional map having

113
00:04:33,620 --> 00:04:31,290
some processing power that we that we

114
00:04:35,180 --> 00:04:33,630
had and putting some batteries and

115
00:04:37,700 --> 00:04:35,190
cameras on there and we sort of followed

116
00:04:39,610 --> 00:04:37,710
the the high-level system requirements

117
00:04:41,930 --> 00:04:39,620
propagated it down to a design and

118
00:04:43,850 --> 00:04:41,940
actually in all reality not all that

119
00:04:46,189 --> 00:04:43,860
surprisingly enough it looks like sort

120
00:04:48,800 --> 00:04:46,199
of a human face when it is flying around

121
00:04:51,379 --> 00:04:48,810
the IFS maybe the reason that our eyes

122
00:04:53,659 --> 00:04:51,389
look that way because it works yes I

123
00:04:56,420 --> 00:04:53,669
mean that if it works I go with it and

124
00:04:58,580 --> 00:04:56,430
the trick of course is how you make this

125
00:05:01,430 --> 00:04:58,590
work can you give us a high-level

126
00:05:03,680 --> 00:05:01,440
explanation of how those goggles take in

127
00:05:08,900 --> 00:05:03,690
information that would help construct

128
00:05:12,890 --> 00:05:08,910
this model yeah so that's a an

129
00:05:14,930 --> 00:05:12,900
interesting question what we do is we we

130
00:05:16,670 --> 00:05:14,940
run through basically an inspection

131
00:05:17,900 --> 00:05:16,680
algorithm where we're for this

132
00:05:20,450 --> 00:05:17,910
particular case we're looking at an

133
00:05:22,010 --> 00:05:20,460
unknown object that we don't really know

134
00:05:23,689 --> 00:05:22,020
much about but we want to build a

135
00:05:27,620 --> 00:05:23,699
three-dimensional model that we can use

136
00:05:30,080 --> 00:05:27,630
for relative navigation so we fly around

137
00:05:33,409 --> 00:05:30,090
the other object taking photos in in

138
00:05:35,659 --> 00:05:33,419
sort of a trajectory and then we match

139
00:05:36,980 --> 00:05:35,669
up what's known as feature points on

140
00:05:38,750 --> 00:05:36,990
each of the object they sort of look

141
00:05:41,240 --> 00:05:38,760
like corners or sharp edges or things

142
00:05:43,399 --> 00:05:41,250
that are very distinguishable that we

143
00:05:45,800 --> 00:05:43,409
can match from frame to frame and from

144
00:05:49,129 --> 00:05:45,810
time step to time step we put that all

145
00:05:51,589 --> 00:05:49,139
through various types of optimization

146
00:05:53,180 --> 00:05:51,599
engines and we can come out with with a

147
00:05:55,939 --> 00:05:53,190
geometric location for each of those

148
00:05:58,550 --> 00:05:55,949
features and start to build up a dense

149
00:06:01,670 --> 00:05:58,560
model a lot like you would have a cam

150
00:06:04,240 --> 00:06:01,680
CAD model of the system from the visual

151
00:06:06,890 --> 00:06:04,250
information that is that is reprojection

152
00:06:11,120 --> 00:06:06,900
rus' that makes some sort of sense it

153
00:06:13,460 --> 00:06:11,130
does and then is the data stored in the

154
00:06:16,279 --> 00:06:13,470
sphere itself or transmitted to some

155
00:06:18,589 --> 00:06:16,289
other other computer that's processing

156
00:06:21,230 --> 00:06:18,599
and putting it together did all of the

157
00:06:24,200 --> 00:06:21,240
processing for this is done on board the

158
00:06:26,930 --> 00:06:24,210
vertigo goggles and that actually led to

159
00:06:28,490 --> 00:06:26,940
led to a fair amount of effort in the

160
00:06:32,930 --> 00:06:28,500
engineering design is that we had to put

161
00:06:35,209 --> 00:06:32,940
a much faster computer on the inside the

162
00:06:37,459 --> 00:06:35,219
goggles than the spheres satellites have

163
00:06:39,279 --> 00:06:37,469
had for years so that computer is

164
00:06:42,519 --> 00:06:39,289
actually a it's kind of a stain

165
00:06:44,170 --> 00:06:42,529
one point two gigahertz Linux computer

166
00:06:46,869 --> 00:06:44,180
that's embedded and customized in a

167
00:06:48,549 --> 00:06:46,879
couple of different ways but but we

168
00:06:50,409 --> 00:06:48,559
store it all locally and we do the

169
00:06:52,689 --> 00:06:50,419
navigation with the respect to that map

170
00:06:55,119 --> 00:06:52,699
all on board the actual satellite and

171
00:06:57,700 --> 00:06:55,129
all autonomously the next step then

172
00:07:00,779 --> 00:06:57,710
would be presumably getting that data to

173
00:07:05,439 --> 00:07:00,789
some piece of hardware that can can

174
00:07:07,869 --> 00:07:05,449
crunch it yes we are so once you

175
00:07:09,820 --> 00:07:07,879
actually have the model on there then

176
00:07:11,670 --> 00:07:09,830
the algorithms are such that the the

177
00:07:13,839 --> 00:07:11,680
goggles can actually crunch on that

178
00:07:16,119 --> 00:07:13,849
crunch on that model and use that for

179
00:07:18,189 --> 00:07:16,129
navigation in high fidelity positioning

180
00:07:20,409 --> 00:07:18,199
and pointing if it wants to go and

181
00:07:23,679 --> 00:07:20,419
rendezvous or darker than interact with

182
00:07:25,779 --> 00:07:23,689
the with the system so it's it's a very

183
00:07:28,959 --> 00:07:25,789
short turnaround it's almost almost a

184
00:07:31,260 --> 00:07:28,969
real-time take a look and make a map yep

185
00:07:33,670 --> 00:07:31,270
and that's that's exactly the idea we

186
00:07:38,639 --> 00:07:33,680
from a high level perspective you don't

187
00:07:40,989 --> 00:07:38,649
want to have the the robot that is doing

188
00:07:43,089 --> 00:07:40,999
interacting with an unknown object in

189
00:07:44,679 --> 00:07:43,099
space having to radio all of its

190
00:07:47,139 --> 00:07:44,689
information down to the ground and then

191
00:07:50,290 --> 00:07:47,149
radio it back up the advantage of having

192
00:07:54,639 --> 00:07:50,300
a fully autonomous closed-loop system is

193
00:07:56,499 --> 00:07:54,649
that it thinks for itself on on on the

194
00:07:59,860 --> 00:07:56,509
fly and even when there's very large

195
00:08:02,019 --> 00:07:59,870
time delays and that that obviously has

196
00:08:03,699 --> 00:08:02,029
appealing characteristics but it it also

197
00:08:05,889 --> 00:08:03,709
makes the problems very hard and that's

198
00:08:08,949 --> 00:08:05,899
that's what the challenge is with this

199
00:08:11,699 --> 00:08:08,959
research why is why is this worthwhile

200
00:08:14,679 --> 00:08:11,709
wise what it would be valuable to have a

201
00:08:17,679 --> 00:08:14,689
free-flying satellite be able to make

202
00:08:19,980 --> 00:08:17,689
such a map there's a lot of there's a

203
00:08:24,029 --> 00:08:19,990
lot of different and very good cases for

204
00:08:27,249 --> 00:08:24,039
for looking at at other objects in space

205
00:08:30,579 --> 00:08:27,259
one of the ones that DARPA is interested

206
00:08:32,679 --> 00:08:30,589
is in going in and recycling old

207
00:08:34,649 --> 00:08:32,689
apertures satellites which are some of

208
00:08:37,749 --> 00:08:34,659
the most expensive parts to launch and

209
00:08:40,990 --> 00:08:37,759
refurbishing them for for new spacecraft

210
00:08:42,579 --> 00:08:41,000
and this is part of a from what I part

211
00:08:43,929 --> 00:08:42,589
of the DARPA Phoenix program where

212
00:08:46,269 --> 00:08:43,939
they're trying to build up a new

213
00:08:48,309 --> 00:08:46,279
satellite by only launching the smallest

214
00:08:50,949 --> 00:08:48,319
elements which is a really cool idea of

215
00:08:52,470 --> 00:08:50,959
basically space recycling as best as I

216
00:08:54,280 --> 00:08:52,480
understand it

217
00:08:57,430 --> 00:08:54,290
there are a couple of other applications

218
00:08:59,500 --> 00:08:57,440
things like going to asteroids which is

219
00:09:02,710 --> 00:08:59,510
sort of a one of the NASA plans and and

220
00:09:04,780 --> 00:09:02,720
sort of understanding how that asteroid

221
00:09:07,510 --> 00:09:04,790
looks as you navigate around it and how

222
00:09:09,250 --> 00:09:07,520
it works even as it's tumbling and

223
00:09:10,810 --> 00:09:09,260
spinning which is one of the the

224
00:09:13,570 --> 00:09:10,820
additional challenges that you have in a

225
00:09:15,760 --> 00:09:13,580
space environment we're showing the NASA

226
00:09:19,060 --> 00:09:15,770
TV audience and video from the test this

227
00:09:21,820 --> 00:09:19,070
morning and seeing one sphere flashing

228
00:09:25,230 --> 00:09:21,830
lights at another one how did the test

229
00:09:27,310 --> 00:09:25,240
go this morning so the hardware and the

230
00:09:30,610 --> 00:09:27,320
operational software part of it went

231
00:09:33,250 --> 00:09:30,620
great the heart it all worked the way

232
00:09:34,900 --> 00:09:33,260
the the system worked when we sort of

233
00:09:38,950 --> 00:09:34,910
handed it over everything checked out

234
00:09:40,870 --> 00:09:38,960
fine and so we have good good certainty

235
00:09:43,690 --> 00:09:40,880
that the hardware is working on orbit

236
00:09:46,300 --> 00:09:43,700
the algorithms that were meant to do the

237
00:09:48,220 --> 00:09:46,310
circumnavigation and inspection worked

238
00:09:49,810 --> 00:09:48,230
pretty well and we could see that they

239
00:09:52,390 --> 00:09:49,820
were they were going along the way they

240
00:09:55,480 --> 00:09:52,400
should but they do need a fair bit more

241
00:09:57,310 --> 00:09:55,490
tuning and refinement but that's exactly

242
00:09:59,019 --> 00:09:57,320
why we do these types of programs with

243
00:10:01,390 --> 00:09:59,029
the International Space Station because

244
00:10:05,040 --> 00:10:01,400
we will be getting the data down as soon

245
00:10:07,150 --> 00:10:05,050
as tomorrow be able to rehash and read

246
00:10:09,280 --> 00:10:07,160
refurbish our algorithms and we're

247
00:10:11,260 --> 00:10:09,290
hoping to operate actually on March

248
00:10:13,390 --> 00:10:11,270
twelfth with a more refined version of

249
00:10:15,519 --> 00:10:13,400
those algorithms and get those I get

250
00:10:17,860 --> 00:10:15,529
some of those milestones checked off so

251
00:10:21,040 --> 00:10:17,870
that's the next step is another another

252
00:10:22,750 --> 00:10:21,050
run just in a couple of weeks yeah it's

253
00:10:24,910 --> 00:10:22,760
it's a very short turnaround time and

254
00:10:26,680 --> 00:10:24,920
that's that's really one of the benefits

255
00:10:29,620 --> 00:10:26,690
of operating with the crew onboard the

256
00:10:32,410 --> 00:10:29,630
International Space Station a dead dr.

257
00:10:34,090 --> 00:10:32,420
Marshburn was helpful oh yeah he was

258
00:10:37,060 --> 00:10:34,100
great I mean I can't say enough good

259
00:10:38,590 --> 00:10:37,070
things about how he how he operated in

260
00:10:41,620 --> 00:10:38,600
this test session he was very very

261
00:10:44,230 --> 00:10:41,630
professional very thorough I mean all

262
00:10:47,380 --> 00:10:44,240
the results said it even it surprised me

263
00:10:48,610 --> 00:10:47,390
how well he did with this hardware that

264
00:10:51,910 --> 00:10:48,620
have never been that has never been

265
00:10:53,230 --> 00:10:51,920
turned on or operated even before one of

266
00:10:56,710 --> 00:10:53,240
the tricky things is that we actually

267
00:11:00,579 --> 00:10:56,720
had to do a recalibration of the cameras

268
00:11:02,260 --> 00:11:00,589
to get to get high quality data and it

269
00:11:04,630 --> 00:11:02,270
was a it was a procedure that we were

270
00:11:05,600 --> 00:11:04,640
sort of holding our breath that we

271
00:11:07,220 --> 00:11:05,610
weren't sure how

272
00:11:09,319 --> 00:11:07,230
easy it would be for the crew to do but

273
00:11:11,210 --> 00:11:09,329
he just nailed it and got it right on

274
00:11:12,710 --> 00:11:11,220
and and we should be able to get good

275
00:11:15,259 --> 00:11:12,720
results on now that we have that

276
00:11:17,300 --> 00:11:15,269
calibration very good Brent will be

277
00:11:19,880 --> 00:11:17,310
looking forward to seeing another run in

278
00:11:22,910 --> 00:11:19,890
a couple of weeks and and and find out

279
00:11:25,009 --> 00:11:22,920
how well it's improved I well we've

280
00:11:27,139 --> 00:11:25,019
learned from the first time yeah now

281
00:11:29,530 --> 00:11:27,149
that we're all definitely very excited

282
00:11:32,090 --> 00:11:29,540
and still plenty of work for us to do

283
00:11:34,040 --> 00:11:32,100
great Brent thank you for your for

284
00:11:37,130 --> 00:11:34,050
helping us learn a little more about

285
00:11:39,259 --> 00:11:37,140
this this morning Brent tweddle is with

286
00:11:40,490 --> 00:11:39,269
the space systems laboratory at the

287
00:11:44,170 --> 00:11:40,500
Massachusetts Institute of Technology